Sound changes that lead to seeing longer-lasting shapes

Published online: 29 January 2018To survive, people must construct an accurate representation of the world around them. There is a body of research on visual scene analysis, and a largely separate literature on auditory scene analysis. The current study follows up research from the smaller literature on audiovisual scene analysis. Prior work demonstrated that when there is an abrupt size change to a moving object, observers tend to see two objects rather than one—the abrupt visual change enhances visible persistence of the briefly presented different-sized object. Moreover, if a sequence of tones accompanies the moving object, visible persistence is enhanced if the tone frequency suddenly changes at the same time that the object’s size changes. Here, we show that although a sound change must occur at roughly the same time as a visual change to enhance visible persistence, there is a fairly wide time frame during which the sound change can occur. In addition, the impact of a sound change on visible persistence is not simply matter of the physical pattern: The same pattern of sound can enhance visible persistence or not, depending on how the pattern is itself perceived. Specifically, a change in a tone’s frequency can enhance visible persistence when it accompanies a visual size change, but the same frequency change will not do so if the shift is embedded in a larger pattern that makes the change merely a continuation of alternating frequencies. The current study supports a scene analysis process that is both multimodal and actively constructive.Support was provided by Ministerio de Ciencia E Innovacion, Grant PSI-2014-53277, Centro de Excelencia Severo Ochoa, Grant SEV-2015-0490, and by the National Science Foundation under Grant IBSS-1519908

The effects of ethnicity, musicianship, and tone language experience on pitch perception

Article information Article has an altmetric score of 3 Free Access Article Information Volume: 71 issue: 12, page(s): 2627-2642 Article first published online: February 16, 2018; Issue published: December 1, 2018 Received: February 21, 2017; Revisions received: September 15, 2017; Accepted: November 25, 2017 Yi Zheng1, Arthur G Samuel1, 2, 3 1Department of Psychology, Stony Brook University, Stony Brook, NY, USA 2Basque Center on Cognition, Brain and Language, Donostia, Spain 3Ikerbasque—Basque Foundation for Science, Bilbao, Spain Corresponding Author: Yi Zheng, Department of Psychology, Stony Brook University, Stony Brook, NY 11794-2500, USA. Email: [email protected] Abstract Language and music are intertwined: music training can facilitate language abilities, and language experiences can also help with some music tasks. Possible language–music transfer effects are explored in two experiments in this study. In Experiment 1, we tested native Mandarin, Korean, and English speakers on a pitch discrimination task with two types of sounds: speech sounds and fundamental frequency (F0) patterns derived from speech sounds. To control for factors that might influence participants’ performance, we included cognitive ability tasks testing memory and intelligence. In addition, two music skill tasks were used to examine general transfer effects from language to music. Prior studies showing that tone language speakers have an advantage on pitch tasks have been taken as support for three alternative hypotheses: specific transfer effects, general transfer effects, and an ethnicity effect. In Experiment 1, musicians outperformed non-musicians on both speech and F0 sounds, suggesting a music-to-language transfer effect. Korean and Mandarin speakers performed similarly, and they both outperformed English speakers, providing some evidence for an ethnicity effect. Alternatively, this could be due to population selection bias. In Experiment 2, we recruited Chinese Americans approximating the native English speakers’ language background to further test the ethnicity effect. Chinese Americans, regardless of their tone language experiences, performed similarly to their non–Asian American counterparts in all tasks. Therefore, although this study provides additional evidence of transfer effects across music and language, it casts doubt on the contribution of ethnicity to differences observed in pitch perception and general music abilities.The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Support was provided by Ministerio de Ciencia E Innovacion (Grant PSI2014-53277), Centro de Excelencia Severo Ochoa (Grant SEV-2015-0490), and the National Science Foundation (Grant IBSS-1519908)

Does seeing an Asian face make speech sound more accented?

Published online: 17 May 2017Prior studies have reported that seeing an Asian face makes American English sound more accented. The current study investigates whether this effect is perceptual, or if it instead occurs at a later decision stage. We first replicated the finding that showing static Asian and Caucasian faces can shift people’s reports about the accentedness of speech accompanying the pictures. When we changed the static pictures to dubbed videos, reducing the demand characteristics, the shift in reported accentedness largely disappeared. By including unambiguous items along with the original ambiguous items, we introduced a contrast bias and actually reversed the shift, with the Asian-face videos yielding lower judgments of accentedness than the Caucasian-face videos. By changing to a mixed rather than blocked design, so that the ethnicity of the videos varied from trial to trial, we eliminated the difference in accentedness rating. Finally, we tested participants’ perception of accented speech using the selective adaptation paradigm. After establishing that an auditory-only accented adaptor shifted the perception of how accented test words are, we found that no such adaptation effect occurred when the adapting sounds relied on visual information (Asian vs. Caucasian videos) to influence the accentedness of an ambiguous auditory adaptor. Collectively, the results demonstrate that visual information can affect the interpretation, but not the perception, of accented speech.Support was provided by Ministerio de Ciencia E Innovacion, Grant PSI2014-53277, Centro de Excelencia Severo Ochoa, Grant SEV-2015- 0490, and by the National Science Foundation under Grant IBSS-1519908

How much do visual cues help listeners in perceiving accented speech?

It has been documented that lipreading facilitates the understanding of difficult speech, such as noisy speech and time-compressed speech. However, relatively little work has addressed the role of visual information in perceiving accented speech, another type of difficult speech. In this study, we specifically focus on accented word recognition. One hundred forty-two native English speakers made lexical decision judgments on English words or nonwords produced by speakers with Mandarin Chinese accents. The stimuli were presented as either as videos that were of a relatively far speaker or as videos in which we zoomed in on the speaker’s head. Consistent with studies of degraded speech, listeners were more accurate at recognizing accented words when they saw lip movements from the closer apparent distance. The effect of apparent distance tended to be larger under nonoptimal conditions: when stimuli were nonwords than words, and when stimuli were produced by a speaker who had a relatively strong accent. However, we did not find any influence of listeners’ prior experience with Chinese accented speech, suggesting that cross-talker generalization is limited. The current study provides practical suggestions for effective communication between native and nonnative speakers: visual information is useful, and it is more useful in some circumstances than others.Support was provided by Ministerio de Ciencia E Innovacion, Grant PSI2014-53277, Centro de Excelencia Severo Ochoa, Grant SEV-2015-0490, and by the National Science Foundation under Grant IBSS-1519908. We also appreciate the constructive suggestions of Dr. Yue Wang and two anonymous reviewers

The Relationship Between Phonemic Category Boundary Changes and Perceptual Adjustments to Natural Accents

published Online First October 21, 2019People often experience difficulties when they first hear a novel accent. Prior research has shown that relatively fast natural accent accommodation can occur. However, there has been little investigation of the underlying perceptual mechanism that drives the learning. The current study examines whether phonemic boundary changes play a central role in natural accent accommodation. Two well-established boundary shifting phenomena were used here—recalibration and selective adaptation—to index the flexibility of phonemic category boundaries. Natural accent accommodation was measured with a task in which listeners heard accented words and nonwords before and after listening to English sentences produced by one of two native Mandarin Chinese speakers with moderate accents. In two experiments, participants completed recalibration, selective adaptation, and natural accent accommodation tasks focusing on a consonant contrast that is difficult for native Chinese speakers to produce. We found that: (a) On the accent accommodation task, participants showed an increased endorsement of accented/ mispronounced words after exposure to a speaker’s accented speech, indicating a potential relaxation of criteria in the word recognition process; (b) There was no strong link between recalibrating phonemic boundaries and natural accent accommodation; (c) There was no significant correlation between recalibration and selective adaptation. These results suggest that recalibration of phonemic boundaries does not play a central role in natural accent accommodation. Instead, there is some evidence suggesting that natural accent accommodation involves a relaxation of phonemic categorization criteria.Support was provided by Ministerio de Ciencia E Innovacion, Grant PSI2017-82563-P, Centro de Excelencia Severo Ochoa, Grant SEV-2015- 0490, by the Basque Government through the BERC 2018–2021 program, and by the National Science Foundation under Grant IBSS-1519908

The effect of orthography on the recognition of pronunciation variants

published Online First October 24, 2019In conversational speech, it is very common for words’ segments to be reduced or deleted. However, previous research has consistently shown that during spoken word recognition, listeners prefer words’ canonical pronunciation over their reduced pronunciations (e.g., pretty pronounced [priti] vs. [priɾi]), even when the latter are far more frequent. This surprising effect violates most current accounts of spoken word recognition. The current study tests the possibility that words’ orthography may be 1 factor driving the advantage for canonical pronunciations during spoken word recognition. Participants learned new words presented in their reduced pronunciation (e.g., [trɒti]), paired with 1 of 3 spelling possibilities: (a) no accompanying spelling, (b) a spelling consistent with the reduced pronunciation (a reduced spelling, e.g., “troddy”), or (c) a spelling consistent with the canonical pronunciation (a canonical spelling, e.g., “trotty”). When listeners were presented with the new words’ canonical forms for the first time, they erroneously accepted them at a higher rate if the words had been learned with a canonical spelling. These results remained robust after a delay period of 48 hr, and after additional learning trials. Our findings suggest that orthography plays an important role in the recognition of spoken words and that it is a significant factor driving the canonical pronunciation advantage observed previously.This work was supported by the National Science Foundation Grant IBSS-1519908. We also acknowledge support provided by Ministerio de Ciencia E Innovacion Grant PSI2017-82563-P, by the Basque Government through the BERC 2018-2021 program, by Economic and Social Research Council (UL) Grant ES/R006288/1 and by Ayuda Centro de Excelencia Severo Ochoa SEV-2015-0490. We thank Marie Huffman and Donna Kat for help with this project

Voices in the mental lexicon: Words carry indexical information that can affect access to their meaning

Available online 11 May 2019The speech signal carries both linguistic and non-linguistic information (e.g., a talker’s voice qualities; referred to as indexical information). There is evidence that indexical information can affect some aspects of spoken word recognition, but we still do not know whether and how it can affect access to a word’s meaning. A few studies support a dual-route model, in which inferences about the talker can guide access to meaning via a route external to the mental lexicon. It remains unclear whether indexical information is also encoded within the mental lexicon. The present study tests for indexical effects on spoken word recognition and referent selection within the mental lexicon. In two experiments, we manipulated voice-to-referent co-occurrence, while preventing participants from using indexical information in an explicit way. Participants learned novel words (e.g., bifa) and their meanings (e.g., kite), with each talker’s voice linked (via systematic co-occurrence) to a specific referent (e.g., bifa spoken by speaker 1 referred to a specific picture of a kite). In testing, voice-to-referent mapping either matched that of training (congruent), or not (incongruent). Participants’ looks to the target’s referent were used as an index of lexical activation. Listeners looked faster at a target’s referent on congruent than incongruent trials. The same pattern of results was observed in a third experiment, when testing was 24 hrs later. These results show that indexical information can be encoded in lexical representations and affect spoken word recognition and referent selection. Our findings are consistent with episodic and distributed views of the mental lexicon that assume multi-dimensional lexical representations.Support for this project was provided by the Spanish Ministry of Science and Innovation, Grant # PSI2014-53277 and # PSI2017-82563- P awarded to A.G.S., the Spanish Ministry of Economy and Competitiveness, Juan de la Cierva-Formación fellowship awarded to E.C.K., and the Spanish Ministry of Economy and Competitiveness, “Severo Ochoa” Programme for Centres/Units of Excellence in R&D (SEV‐2015‐490)

Reconciling the Contradictory Effects of Production on Word Learning: Production May Help at First, but It Hurts Later

Published: March 2022Does saying a novel word help to recognize it later? Previous research on the effect of production on this aspect of word learning is inconclusive, as both facilitatory and detrimental effects of production are reported. In a set of three experiments, we sought to reconcile the seemingly contrasting findings by disentangling the production from other effects. In Experiment 1, participants learned eight new words and their visual referents. On each trial, participants heard a novel word twice: either (a) by hearing the same speaker produce it twice (Perception-Only condition) or (b) by first hearing the speaker once and then producing it themselves (Production condition). At test, participants saw two pictures while hearing a novel word and were asked to choose its correct referent. Experiment 2 was identical to Experiment 1, except that in the Perception-Only condition each word was spoken by 2 different speakers (equalizing talker variability between conditions). Experiment 3 was identical to Experiment 2, but at test words were spoken by a novel speaker to assess generalizability of the effect. Accuracy, reaction time, and eye-movements to the target image were collected. Production had a facilitatory effect during early stages of learning (after short training), but its effect became detrimental after additional training. The results help to reconcile conflicting findings regarding the role of production on word learning. This work is relevant to a wide range of research on human learning in showing that the same factor may play a different role at different stages of learning.Support for this project was provided by the Spanish Ministry of Science and Innovation, Grant PSI2017-82563-P, awarded to Arthur G. Samuel and by the Spanish Ministry of Economy and Competitiveness through the Juan de la Cierva-Formación fellowship, FJCI-2016–28019, awarded to Efthymia C. Kapnoula. This work was partially supported by the Basque Government through the BERC 2018–2021 and BERC 2022–2025 programs, and by the Spanish State Research Agency through BCBL Severo Ochoa excellence accreditation SEV-2015-0490 and CEX2020-001010-S. This project has received funding from the European Union’s Horizon 2020 research and innovation program, under the Marie Skłodowska-Curie grant agreement 793919, awarded to Efthymia C. Kapnoula

Better than native: Tone language experience enhances English lexical stress discrimination in Cantonese-English bilingual listeners

Available online 13 April 2019While many second language (L2) listeners are known to struggle when discriminating non-native features absent in their first language (L1), no study has reported that L2 listeners perform better than native listeners in this regard. The present study tested whether Cantonese-English bilinguals were better in discriminating English lexical stress in individual words or pseudowords than native English listeners, even though lexical stress is absent in Cantonese. In experiments manipulating acoustic, phonotactic, and lexical cues, Cantonese-English bilingual adults exhibited superior performance in discriminating English lexical stress than native English listeners across all phonotactic/lexical conditions when the fundamental frequency (f0) cue to lexical stress was present. The findings underscore the facilitative effect of Cantonese tone language experience on English lexical stress discrimination.This article is, in part, based on the fourth chapter of the PhD thesis submitted by William Choi to The University of Hong Kong. This research was supported, in part, by the Language Learning Dissertation Grant from Language Learning to William Choi. It was also supported by the Pilot Scheme on International Experiences for Research Postgraduate Students from The University of Hong Kong to William Choi, and the Early Career Scheme (27402514), General Research Fund (17673216), and General Research Fund (17609518) from the HKSAR Research Grant Council to Xiuli Tong. Support was also provided by Ministerio de Ciencia E Innovacion, Grant PSI2014-53277, Centro de Excelencia Severo Ochoa, Grant SEV-2015-0490, and by the National Science Foundation under Grant IBSS-1519908 to Arthur Samuel. We thank Benjamin Munson for his useful suggestion about the syllable-timed nature of Cantonese and the four anonymous reviewers for comments that have helped us to develop our ideas and presentation more clearly